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^ ' It is shown that any communication system which admits a sum-product algorithm (SPA) based 

receiver also admits a corresponding linear-programming (LP) based receiver. The two receivers have a 
relationship defined by the local structure of the underlying graphical model, and are inhibited by the 
same phenomenon, which we call pseudoconfigurations. This concept is a generalization of the concept 
of pseudocodewords for linear codes. It is proved that the LP receiver has the 'maximum likelihood 
certificate' property, and that the receiver output is the lowest cost pseudoconfiguration. Equivalence 
of graph-cover pseudoconfigurations and linear-programming pseudoconfigurations is also proved. A 
concept of system pseudodistance is defined which generalizes the existing concept of pseudodistance 



for binary and nonbinary linear codes. While the LP receiver generally has a higher complexity than the 
corresponding SPA receiver, the LP receiver and its associated pseudoconfiguration structure provide a 
useful tool for the analysis of SPA receivers. As an illustrative example, we show in detail how the LP 
design technique may be applied to the problem of joint equalization and decoding of coded transmissions 
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over a frequency selective channel. A simulation-based analysis of the error events of the resulting LP 



receiver is also provided. 
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I. Introduction 

The decoding algorithms for some of the best known classes of error-correcting code to date, namely 
concatenated ("turbo") codes |1] and low-density parity check (LDPC) codes [2], have been shown to 
be instances of a much more general algorithm called the sum-product algorithm (SPA) (3j , (U, (5]. 
This algorithm solves the general problem of marginalizing a product of functions which take values in 
a semiring 1Z. In the communications context, 1Z is equal to R and the maximization of each marginal 
function minimizes the error rate for each symbol, under the assumption that the system factor graph is a 
tree [5 ]. It has been recognized that many diverse situations may allow the use of SPA based reception [6], 
including joint iterative equalization and decoding (or turbo equalization) joint iterative equalization 
and multiuser detection (MUD) (8], and joint source-channel decoding |9). 

Recently, a linear-programming (LP) based approach to decoding linear (and especially LDPC) codes 
was developed for binary iTTOl . iTTTTl and nonbinary coding frameworks lfl2ll . |[T3l . The concept of 
pseudocodeword proved important in the performance analysis of both LP and SPA based decoders 
lfl4ll . lfT3l . iflol . Also, linear-programming decoders for irregular repeat-accumulate (IRA) codes and 
turbo codes were described in [17]. While the complexity of LP decoding is conjectured to be higher 
than for SPA decoding, the LP decoder has many analytical advantages, such as the property that a 
codeword output by the LP is always the maximum likelihood (ML) codeword, and the equivalence of 
different pseudocodeword concepts in the LP and SPA domains [Til . lfT3l . For the case of LDPC codes, 
tight connections were observed between the LP decoding and min-sum decoding frameworks |fl8l . 

Recently, some authors have considered use of similar linear-programming techniques in applications 
beyond coding. An LP-based method for low-complexity joint equalization and decoding of LDPC coded 
transmissions over the magnetic recording channel was proposed in |fT9~l . In this work, the problem of 
ML joint detection, which may be expressed as an integer quadratic program, is converted into a linear 
programming relaxation of a binary-constrained problem. In the case where there is no coding, it was 
shown in [19] that for a class of channels designated as LP-proper channels, the LP solution matches 
the ML solution at all values of signal to noise ratio (SNR); however, for some channels which are not 
LP-proper, the system evinces a frame error rate (FER) floor effect. The work of 1201 considered an 
LP decoder which incorporates nonuniform priors into the original decoding poly tope of [11], and also 
application of an LP decoder to transmissions over a channel with memory, namely the non-ergodic Polya 
channel. In both |fl9l and [20], performance analysis proved difficult for the case where the channel has 
memory. 



In this paper it is shown that the problem of maximizing a product of 7^-valued functions is amenable to 
an approximate (suboptimal) solution using an LP relaxation, under two conditions: first, that the semiring 
1Z corresponds to a subset of R under real addition and multiplication, and second, that all factor nodes 
of degree greater than one are indicator functions for a local behavior. Fortunately, these conditions are 
satisfied by a large number of practical communication receiver design problems. Interestingly, the LP 
exhibits a "separation effect" in the sense that degree-1 factor nodes in the factor graph contribute the 
cost function, and the remaining nodes determine the LP constraint set. This distinction is somewhat 
analagous to the case of SPA-based reception where degree-1 factor nodes contribute initial messages 
exactly once, and all other nodes update their messages periodically. Our LP receiver generalizes the LP 
decoders of iPTTl . lTl3l . iTTTl . A general design methodology emerges, parallel to that of SPA receiver 
design: 

1) Write down the global function for the transmitter-channel combination. This is a function propor- 
tional to the probability mass function of the transmitter configuration conditioned on the received 
observations. 

2) Draw the factor graph corresponding to the global function. 

3) Read the LP variables and constraints directly from the factor graph. 

The proposed framework applies to any system with a finite number of transmitter configurations, and 
also allows for treatment of "hidden" (latent) state variables (as was done for the SPA receiver case in 
0). It allows for a systematic treatment of variables with known values (e.g. known initial/final channel 
states or pilot symbols). Incorporation of priors for any subset of transmitter variables is straightforward, 
and thus the polytopes of |[20l Section II] follow as simple special cases of our framework. Our framework 
allows derivation of LP decoders also for tail-biting trellis (TBT) codes; in this case the relevant pseudo- 
configurations correspond to the TBT pseudocodewords as defined in lfl4l . It is proved that the LP receiver 
error events, which we characterize as a set of linear-programming pseudoconfigurations, are equivalent 
to the set of graph-cover pseudoconfigurations, which are linked to error events in the corresponding 
SPA receiver. Furthermore, we define a general concept of pseudodistance for the transmission system, 
which generalizes the existing concept of pseudodistance for binary and nonbinary linear codes to the 
case of the general LP receiver. 

In order to illustrate the LP receiver design methodology outlined above, we provide a step-by-step 
derivation of an LP receiver which performs joint equalization and decoding of coded transmissions over 
a frequency selective channel. We then provide a simulation study of a simple case of this receiver, 



including error rate results and pseudodistance spectra, together with a complete description of error 
events at low values of pseudodistance. Performance results are also presented for a low-density code 
transmitted over an intersymbol interference channel. We note that a similar line of work is considered 
in E71 ; the LP presented therein is equivalent to the one we derive in Section IVIH except that it does 
not deal with the case of known states in the trellis. 

This paper is organized as follows. Section JI] introduces the general problem to be solved, along 
with appropriate notations, and Section [III] develops a general linear program which solves this problem. 
Section [IV] introduces an efficient linear program which provides a suboptimal solution, and Section 
IVl develops an equivalent program with a lower description complexity. Section [VT] introduces general 
concepts of system pseudoconfigurations and pseudodistance. Section[VTI]provides a detailed development 
of an LP receiver which performs joint equalization and decoding, and Section IVIIII presents a detailed 
simulation-based analysis of this receiver for the case of binary-coded transmissions over the Proakis B 
channel. 

II. Problem Statement and Notations 

We begin by introducing some definitions and notation. Suppose that we have variables X{, i G I, 
where X is a finite set, and the variable Xi lies in the finite set A% for each i £ I. Let x = (x^jj]; then 
x is called a configuration, and the Cartesian product A = Yliex^i * s called the configuration space. 
Suppose now that we wish to find the configuration x G A which maximizes the product of real-valued 
functions 

u{x)=Hf j (x j ) (i) 

where J is a finite set, Xj = (xi)i e x 3 and Ij C X for each j G J. We define the optimum configuration 
x ov x, to be the configuration x G A which maximizes^ (Q]). The function u{x) is called the global 
function |5|. In the communication receiver design context, the global function is taken to be any function 
proportional to the probability mass function of some set of transmitter-channel variables (information 
bits, coded symbols, state variables etc.) conditioned on the receiver observations. Maximization of the 
global function therefore corresponds to maximum a posteriori (MAP) reception. 



'All vectors in the paper are row vectors; also, the notation (vt)teT denotes a vector whose entries are equal to {vt : t £ T} 
with respect to some fixed ordering on the elements of T. 

2 In most cases of practical importance, a single x opt maximizes |QJ with probability 1; henceforth, we will assume a unique 



The factor graph for the global function u(x) and its factorization (Q]) is a (bipartite) graph defined 
as follows. There is a variable node for each variable x% (i G T) and a factor node for each factor fj 
(j G J). An edge connects variable node xi to factor node fj if and only if Xi is an argument of fj. 
Note that for any j G J , lj is the set of i G 1 for which Xj is an argument of fj. Also, for any i Gl, 
the set of j G J for which x» is an argument of fj is denoted Ji. The degree of a node t>, denoted d(v), 
is the number of nodes to which it is joined by an edge. 

Let C C J denote the set of all j G J such that factor node fj is an indicator function for some local 
behavior Bj, i.e., 

fj( X j) = I(xj eBj) VjG£ (2) 

where the indicator function for the logical predicate a is defined by 

{1 if a is true 
otherwise. 

In the communication receiver design application, the set C comprises constraints such as parity-check 
constraints and state-space constraints, and also may account for variables with known values (pilot 
symbols, known states). 

Note that we write any v G Bj as v = (vi)i & x 3 , i.e., v is indexed by lj. Also we define the global 
behavior B as follows: for any x G A, we have x G B if and only if Xj G Bj for every j G C. The 
configuration x G A is said to be valid if and only if x G B. 

Next define 

y = {i G 1 : 3j G • 

We assume that for every j G Ji\C, the factor node fj has degree equal to one. This allows us to define, 
for each i G y, 

hi {xi) = J] fj (x^ . 
jeJi\C 

In the communication receiver design context, the set y corresponds to the set of observables, i.e. the 
set of variables for which noisy observations are available. 
So, without loss of generality we may write 

u(x) = l[h i (x l )-l[fj(xj) . (3) 

We assume that the function hi (xi) is positive-valued for each i £ y. Also, denoting the Cartesian 
product Ay = Y\i<=yAi, we define the projection 

Py : A — V Ay such that Py (x) = (xi)i e y . 



Also, we adopt the notation xy = (xi)i & y for elements of Ay. 

We assume that the mapping Py is injective on B, i.e., if Xx,x% G B and Py(x\) = Py{x2), 
then x\ = X2- This corresponds to a 'well-posed' problem. Note that in the communication receiver 
design context, observations (or "channel information") may only be contributed through the nodes x^ 
for i G y. Therefore, failure of the injectivity property in the communications context would mean that 
one particular set of channel inputs could correspond to two different transmit information sets, which 
would reflect badly on system design. 

III. Maximization of the Global Function by Linear Programming 
Using (f2]) and (O, we may write 



which acts as an 'indicator vector' for the value a G A{. Building on this, for xy G Ay we define 
the indicator vector S(xy) = (£j(xj))j e ;y of length Ny = ^2 i£ y Ni, which is a concatenation of the 
individual indicator vectors for each of the elements of xy. It is easy to see that H is an injective function 
on Ay. 

Now, for vectors g G R Ny we adopt the notation 



In particular, we define the vector A G M. Ny by setting A 4 = log/ij(a) for each i G y, a G A{. This 
allows us to develop the formulation of the optimum configuration as 



Note that the optimization has reduced to the maximization of an inner product of vectors, where the 
first vector derives only from observations (or "channel information") and the second vector derives only 
from the global behavior (the set of valid configurations). This problem may then be recast as a linear 
program LP1 as shown below. 




For each i G X, a G A4, let £j(a) 




9 = {9i)iey where g i = {gf') a£ A, Vi G y . 





LP1: Optimum Configuration 




x op t = Py 1 {Er l (g ovt )) 


(4) 


where 




ffopt = arg max \g T 


(5) 


and the maximization is over the convex hull of all points corresponding 


' to valid configurations: 


Ky{B) = conv{H (Py (x)) : x G B) . 


(6) 



IV. LP Relaxation 

In order to reduce the description complexity of LP1, we introduce auxiliary variables whose con- 
straints, along with those of the elements of g G W Ny , will form the relaxed LP problem. We introduce 
auxiliary variables pj^ for each j G C, b G Bj, and we form the vector 

P = (Pj)jec where Pj = (Pj,b) beBj Vj G £ • 

Also, for vectors g G M N where N = X^ex-^i' we denote g = (g^i^x and g = (g^i^y, where 
9i = w )a£Ai f° r ea ch i G Z. For x G A we define the indicator vector H(;r) = (^(xj))^ of length 
N. The function H is injective on A. 

The new LP optimizes the cost function Xg T over the polytope Q defined with respect to variables g 
and p, as shown in the following. 



LP2: Efficient Relaxation 




Cost Function: Xg T = E g logY\ ie y /ij(xj) 




Constraints (Polytope Q): 




V? G £, V6 G Bj, Pj , b >0, 


(7) 


Vj G £, ^ = 1 , 


(8) 






Vj G £, Vi G Zj, Vq G A, 5* = ^ ' 


(9) 







Note that fTJ and ([8]) imply that we may view x G £> as a random vector, and for each j G £ the 
vector p- may be interpreted as a probability distribution on the local configuration xj G Bj; © then 
expresses each vector g { (for each i £ I) as the induced probability distribution on X{ G Ai. It may be 



easily checked that Xg T is then the expectation of log JTjey hi( x i) w i tn respect to this distribution^; this 
interpretation of the cost function will be useful in our treatment of system pseudodistance in Section 
IVll A similar probabilistic interpretation was also considered in the context of pseudocodewords of 
graph-cover decoding in |[T4l^ . 

If the LP solution (g out ,p) is an integral point in Q (i.e. all of its coordinates are integers), the receiver 
output is the configuration x out = Py 1 (s ^(g ou t)\ (we shall prove in the next section that this output 
is indeed in B). Of course, in the communications context, we are usually only interested in a subset of 
the configuration symbols, namely the information bits. If the LP solution is not integral, the receiver 
reports a 'receiver failure'. 

V. Efficient Linear-Programming Relaxation and its Properties 

We next define another linear program, and prove that its performance is equivalent to that defined in 
Section |IV] This new program achieves lower description complexity than LP2 by removing unnecessary 
constraints from the formulation. We remove constraints in two ways: from the variable set, and by 
defining constraints with respect to an 'anchor node'. 

For each i 6l, let a, be an arbitrary element of Ai, and let A~ = Ai\{cti} (note that for each i G X, 
\Ai\ > 2, otherwise Xi is not a 'variable'). For each i G X, a G Ai, define £j(a) = (1(7 = a)) 1&A - . This 
indicator vector of length N± = \A~\ is the same as £j(a) except that the entry corresponding to ai has 
been removed. Correspondingly, for each xy G Ay we define the indicator vector 3(xy) = (€i(xi))n=y 
of length Ny = J2iey^i~- Ag am > the mapping S is injective. 

Now, for vectors g G R N y , we adopt the notation 

9 = (9i)iey where g { = (g^)^- Vi G y . 

In particular, we define the vector A G M N y by setting \\ a ^ = log \hi{a) / hi(pn)] for each i G y, a G A~. 
For each % G X\y, let tj be an arbitrary element of Jj. The LP is then as follows. 

3 lf g represents a probability distribution on x, we denote the expectation of F(x) with respect to the distribution g as 

EgF( X ). 

4 Another interpretation of the polytope Q is that the projection of Q onto g is formed by the intersection of convex hulls 
corresponding to the local behaviors, i.e. (g^iex* € conv{H(Bj)} for all j 6 C. 



LP3: Low-complexity Relaxation 
Cost Function: \g T 

Constraints (Poly tope Q): Constraints © and ([8]), together with 




Vz G y, Vj G JiDC, Vq G A~, 






(10) 


and 




VieZW, VjG Ji\{ti}, VaGir, 




E6eB j :6 i =a?'i,b = ^2beB ti :bi=aPU,b ■ 


(11) 



The receiver output is equal to the configuration x out = S (fl ou t) G £> in the case where the solution 
(fl out ,p) to LP3 is an integral point in Q, and reports a 'receiver failure' if the solution is not integral. 

The following theorem ensures the equivalence of the linear programs LP2 and LP3, and also assures 
the optimum certificate property, i.e., if the output of either LP is a configuration, then it is the optimum 
configuration. In the communications context, the optimum corresponds to the maximum likelihood 
transmit configuration; thus in this case we have the maximum likelihood certificate property. 

Theorem 5.1: The two linear programs LP2 and LP3 produce the same output (configuration or 
receiver failure). Also, if the receiver output is a configuration, then it is the optimum configuration, i.e., 

■"out — 3?opt- 

Proof: It is straightforward to show that the mapping 



V : Q 



Q 



(a,p) ^ (g,p) 



defined by 



and with inverse 



9; 



(a) 



~g\ a) if i G y, a G A~ 

1 ~ E/3e.4r ^ it i ^y,a = ai 
Eb£B ti :h=aPU,b if i G T\y 



9, 



(«) _ » 



9, 



is a bijection from one polytope to the other (i.e. g G R y satisfies the constraints of LP3 for some 
vector p if and only if g G 1^ with (g,p) = V(g,p) satisfies the constraints of LP2 for the same 



vector p). Also 

= E ( E log/iiCa^-log^CaOIl-^H] 

= Xg T -J2 l °Ehi(ai) , (12) 

implying that the bijection V preserves the cost function up to an additive constant. 

Next, we prove that for every configuration x G B, there exists p such that (S (x) ,p) G Q. Let x £ B, 
and define 

1 if b = (xi) ieX] 
otherwise. 

Letting g = S(Vy(x)) and g = S(x), it is easy to check that (g,p) G Q and (</,p) G Q (and that in 
fact (g,p) = V(g,p)). This property ensures that every valid configuration x G B has a "representative" 
in the polytope, and thus is a candidate for being output by the receiver. 

Next, let (g,p) G Q and let g G M N be such that (g,p) = V(g,p) G Q. Suppose that all of the 
coordinates of p are integers. Then, by (O and ([8]), for any j G C we must have 

1 if 6 = &C) 



Vje£,beBj, p jjb 



V6 G Bj, p 3) 6 
for some 



otherwise 



Next we note that for any i G 1, j, k G D £, if 6-^ = a then (using Q) 



E ^ 5 = 1 = E pfc > 5 ( 13 ^ 



(a) 
ffi = 

beBi-bi=a beB k :bi=a 



(k) 

and thus 6- = a. Therefore, there exists x G A such that 

(a?i) i6 2i = b U) Vj G £ . 
Therefore, a; is a valid configuration (a; G B). Also we may conclude from (TT3l) that 

(a) 1 if X i = a 

9i =< 

I otherwise 

and therefore g = S (x). Also, from the definition of the mapping V, we have g = S (Py (a;)). 

Summarizing these results, we conclude that (g opt ,p) G Q optimizes the cost function \g T over Q 
and is integral if and only if (g opt ,p) = V^ 1 (g opt ,p) G Q optimizes the cost function \g T over Q 



and is integral, where S (g) = x £ B and xy = S (g) = Vy(x). A graphical illustration of these 
relationships is shown in Figure Q] ■ 

Thus both LP receivers output either the optimum configuration or a receiver failure, and have the same 
performance. LP3 has lower complexity and is suitable for implementation (we shall use it to solve the 
joint equalization and decoding problem in Section I VIII ); however, for theoretical work LP2 is more 
suitable (we shall use this LP throughout Section IVD ). 

VI. PSEUDOCONFIGURATIONS 

In this section, we show a connection between the failure of the LP and SPA receivers based on 
pseudoconfiguration concepts, and define a general concept of pseudodistance for LP receivers. 

A. Linear-programming pseudoconfigurations 

Definition 6.1: A linear-programming pseudoconfiguration (LP pseudoconfiguration) is a point (g, p) 
in the polytope Q of the linear program LP2. 

B. Factor graph covers and graph-cover pseudoconfigurations 

We next define what is meant by a finite cover of a factor graph. 

Definition 6.2: Let M be a positive integer, and let M. = {1, 2, • • • , M}. Let Q be the factor graph 
corresponding to the global function u and its factorization given in £[]). A cover configuration of degree 
M is a vector x^ M ^ = (x\ )iex where x\ = (xj )m ) mG x G Af' 1 for each i€l Define u^ M ^ as the 
following function of the cover configuration x^ M > of degree M: 

U (M) ^(M)) = JJ JJ f. {x . m) (14) 

where, for each j € S, i € Xj, IL;^ is a permutation on the set A4, and for each j € J\ m G .M, 

■Kj'jm = (- c i ) ri3 J i(m))i6lj ■ 

A cover of the factor graph Q, of degree M, is a factor graph for the global function u^ M ^ and its 
factorization (fT4l) . In order to distinguish between different factor node labels, we write ([141) as 

U (M) ^(M)) = JJ JJ /. m (a . . m ) 

m£M jej 

where / JiTn = fj for each j £ J, m £ M. 



It may be seen that a cover graph of degree M is a graph whose vertex set consists of M copies of xi 
(labeled Xi >m ) and M copies of fj (labeled fj >m ), such that for each j G J, i G Tj, the M copies of Xi 
and the M copies of fj are connected in a one-to-one fashion determined by the permutation {IL;^}. 

We define the cover behavior Bm as follows. The cover configuration x^' 1 ^ lies in Bm if and only if 
Xj >m G Bj for each j G J, m G M. 

For any M > 1, a graph-cover pseudoconfiguration is defined to be a valid cover configuration (i.e. 
one which lies in the behavior Bm)- 

For any graph-cover pseudoconfiguration, we also define the (unsealed) graph-cover pseudoconfigura- 
tion vector h G M. N according to 

h = (hi) ie x where K = (h^ ) a&Ai Vie! 

and 

h i =\{me M: Xi t7n = a}\ 

for each i £ 1, a £ Ai. Finally, we define the normalized graph-cover pseudoconfiguration vector 
g G R N by g = h/M. 

C. Equivalence between pseudoconfiguration concepts 

In this section, we state the equivalence between the set of LP pseudoconfigurations and the set of 
graph-cover pseudoconfigurations. The result is summarized in the following theorem. 

Theorem 6.1: There exists an LP pseudoconfiguration (g,p) if and only if there exists a graph-cover 
pseudoconfiguration with normalized pseudoconfiguration vector g. 

The proof of Theorem 16. II follows the lines of the proof of [13, Theorem 7.1]; the details are omitted. 

D. Pseudodistance 

In this section we define the concept of system pseudodistance for communication systems where 
the set of variables {xi}i<=y is observed through complex additive white Gaussian noise (AWGN). For 
each i £ y, we have an observation yi = Pi + iq% which is formed by passing the symbol xi through 
a modulation mapper and adding complex Gaussian noise with variance a 2 per real dimension (here 
i = yj— 1). The mapper operates according to the following rule: for i G y, a G A% is mapped to 
Si(a) = di{a) + ibi(a) . Note that this includes cases where different symbols may use different mappers, 
e.g. orthogonal frequency division multiplexing (OFDM) systems with adaptive modulation. Then 



In what follows, we denote the transmitted and received vectors by s(xy) = (sj(xj))j 6 y and y = (yi)i<=y 
respectively. 

Suppose that the actual transmitter configuration is x € £>, and let w = S(Vy(x)). The LP receiver 
LP2 favours the pseudoconfiguration K = (ff,p)eS over x if and only if \g T > Xw T , i.e. if and only 
if 

E 9 log Y\ hi(xi) > E w log Y\ hi(xi) 
Using (fT5l) . this condition is easily seen to be equivalent to 

E g \\y - s(xy)\\ 2 < E w \\y - s(x y )\\ 2 = \\y - s(x y )\\ 2 . 
Using E g ||y|| 2 = ||y|| 2 , this may be rewritten as 

(MiPi + N iqi ) > R 

iey 

where M { = 2 (E g a^Xi) - aj(xj)), iVj = 2 (E g bjfo) - 6* and 

R = E g \\s{xy)f-\\s{xy)\\ 2 . 

In the absence of noise, the modulated signal point in the signal space with 2\y\ dimensions and coordi- 
nates {pi}iey and {qi}i^y is given by p, = aj(xj) and qi = bi(xi) for all i £ y. The squared Euclidean 
distance from this point to the plane Yliieyi^-iPi + ^iQi) = -R is then given by D 2 = (R — S) 2 /V, 
where 

3 = 2^2 [ai{xi)E g (n(xi) + bi(xi)E g - 2||s(%)|| 2 

and 

V = A\\E gS (xy)-s(xy)\\ 2 . 

Thus the effective Euclidean distance or system pseudodistance between the configuration x £ 13 and the 
pseudoconfiguration k = (g,p) e Q is 

d off (x,K) = 2£> = * " k yj . (16) 



Z^iey [\ s i( x i)\ 2 + EaeA (l s i( a )l 2 ~ 2ai(a)ai0i) - 26 i (a)6 i (a;i)) 5. 



(a) 



(17) 



This generalizes the concept of pseudodistance given in lfl4l and ||2T1 for binary and nonbinary codes, 
and in particular generalizes |[T4l Theorem 2.1] which was proved for real AWGN and pseudocodewords 



of a balanced computation tree for a nonbinary code. Note that the system pseudodistance depends on 
the transmitter configuration (i.e. the information word); while it was proved in [11] for binary codes and 
in ll22ll for nonbinary codes that under a sufficent channel symmetry condition the performance of LP2 is 
independent of the codeword transmitted, this property does not hold in the current more general context. 
The pairwise error probability between the transmitter configuration x G B and the pseudoconfiguration 
n G Q is given by 



P e (x, k) = Q 



d c s(x,K) 
2a 



(18) 



where Q(x) = ^ exp(— 1 2 /2) dt denotes the Gaussian Q-function. We define the minimum pseu- 
dodistance of the system as 



jmm 
"eflf 



min d e g(x, k) 

xeB,neQ 



The minimum pseudodistance provides a first-order measure of system performance. Finally, it is straight- 
forward to show that in the case where the pseudoconfiguration k G Q corresponds to a configuration 
z G B, we have g = S(Vy(z)) and (TTTT ) reduces to 

des(x, K) = \\s(Py(z)) - s(xy)\\ 

which is the ordinary Euclidean distance between the two relevant modulated signals. 

For the case of real AWGN with variance a 2 per dimension (here m and Si(a) are real for each i G y), 
a similar analysis shows that assuming the transmitter configuration is x G B, the probability of error 
due to pseudoconfiguration K = (j,p)eQ is again given by (TT8T ). where 



d eS (x,K) 



(a) 



IE 



tf + V4- 2ti1Tli)\ 



Si{a)g. 
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Si, [ X% 



EiGV ( 



(19) 



mi 



where we define t = (t{) ie y, m = (rrii) iE y and v = (vi) ie y, and for each i G y we have t{ = S{(xi), 



mi = E g si(xi) = ^2 s i( a )9i 
aeAi 



(a) 



and 



aeAi 



VII. Example Application: LP-Based Joint Equalization and Decoding 

In this section we consider an example application where we use the above framework to design an LP 
receiver for a system using linear coding and memoryless modulation over a frequency selective channel 
with AWGN. 

A. System model and notation 

The system model may be described as follows. Information-bearing data are encoded to form code- 
words of the (binary or nonbinary) code C over the ring 9\, characterized by the mxn parity-check matrix 
H = {Hj : i) over IH. Denote the set of coded symbol indices and parity-check indices by U = {1, 2, • • ■ ,n} 
and V = {1, 2, • • • , m} respectively. For each j G V, define the j-th local code over 9t by 

Cj = {(ci)i eUj : ^2 H 3> iCi = °} 

where Uj C U is the support of the j-th row of H for each j G V, and multiplication and addition are 
over 9\. Thus c G C if and only if Cj = (cj)j 6 ^. lies in Cj for each j G V. 

Each coded symbol Cj is mapped directly to a modulation symbol x% = G T, where T C C 

denotes the transmit constellation. The (injective) modulation mapping is defined by X : 9i — > T. 
The modulated symbols are transmitted over a (possibly time-variant) frequency selective channel with 
AWGN; the received signal is given by 

L 

yi = y~] hf'xi-t + rii 
t=o 

where m is a zero-mean complex Gaussian random variable with variance a 2 . We assume that the receiver 
has complete knowledge of the set of complex channel coefficients j j . 

We adopt a state-space (trellis) representation for the channel with state space S = 9\ L ; also let S~ = 
<S\{0ixl}- The local behavior (trellis edge set) for the state-space model is denoted by V. For d G V, let 
ip(d), opj(<i), s s (d) and s E (d) denote the channel input, output (at time index i), initial state and final 
state respectively. Thus if we set V = 1H L+1 and adopt the notation d = (do d\ • • • di,) G V, we may 
have ip(cZ) = d , s s (d) = (di d 2 ■■■ d L ), s E (d) = (d d 1 • • • d L - X ), and op^d) = Y,t=o ht^X(d t ). 
Also let Z>- = Z>\{0 lx(i+1) }. 

Finally, we note that the more common case where D\ is a finite field is included as a special case of 
this framework, and also that this system is a generalization of the system of [13] to frequency selective 
channels. 



B. Factor graph and linear-programming receiver 

We next derive the factor graph for the communication problem. Denote the state sequence followed by 
the channel by s = (so si • • ■ s n ), and the corresponding sequence of trellis edges by d = (d\ d 2 • • • d n ), 
where di = (q Cj_i • • • Cj_^) for i £lA. For the purpose of exposition we assume that the final channel 
state s n is unknown to the receiver, but that the initial channel state so is known to the receiver and is 
OixL- We assume that each codeword in C is transmitted with equal probability. Using Bayes' rule, the 
a posteriori probability of the transmitter-channel configuration conditioned on the entire received data 
is given by (here P denotes probability, and p denotes probability density) 

P(c,s,d\y) = Piyld)P{C > S > d) 

p(y) 

Thus the global function is given b)|j 

u(c,s,d) = J]_Qi(di) ■ Y[xj(cj) ■ \\Ti(ci,di,Si-i,Si) ■ u(s ) (20) 
ieu jev ieu 

Here Qi(di) = p(yi\di) for each i £W, and Xji c j) = I( c j £ Cj) f° r eacn 3 £ V. The factor for each 

i € U represents the channel state-space constraint and may be written as Tj(c, d, s^) = I(c = 

do) ■ I(s^ = (di di ■ ■ ■ di)) ■ I(s^ = (do di ■ ■ ■ dL-i))- The factor u(so) = I(sq = 0\xl) expresses 

the receiver's knowledge of the initial state of the channel. 

The factor graph corresponding to the global function and its factorization given by (l20l is illustrated 

in Figure |2] for n = 7, m = 3, and the binary [7, 4] Hamming code. Here the set of indicator function 

type factor nodes is C = {u, %i, \%i X3> T\,T2, ■ ■ ■ ,7V}, and the set of "observable" variable nodes is 

y = {d 1 ,d 2 ,--- ,d 7 }. 

The LP is then derived using the rules defined in Section [V] After some simplification^], this reduces 
to the following; each constraint is marked with the corresponding constraint from LP3 from which it 
derives. Here U~ = U\{n} and £H _ = 9l\{0}. 



5 For a system with transmitter-channel configurations x G X and received observations y, setting u(x) = K ■ P(x\y) (where 
K does not depend on x) implies that the receiver decision rule x = arg m&xxsA u(x) minimizes the configuration error 
probability. 

6 Consisting primarily of the elimination of variables and g[ K 



LP4: Joint Equalization and Decoding 






Cost Function: 
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where we nave, tor i £ U, a £ TJ , 
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(22) 


Constraints (Poly tope Q): 
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(25) 


together with 






Vi e W, Vj e W if Vr € jr- E 

deO:ip(d)=r 


= E w i> b 

beCf. 6i=r 


(26) 


and 






Vs e 5", ^ ?i,d = 

deV:s s (d)=s 





(27) 


from ([TTb. 







LP4 is capable of joint equalization and decoding, and has strong links (via Theorems 15.11 and 16.11 ) 
to the corresponding "turbo equalizer" based on application of the sum-product algorithm to the same 
factorization of the global function. Finally, note that in the case where the receiver output is integral, 
the LP variables and {q^d} serve as indicator functions for the local codewords and the trellis 

edges respectively. 

C. Low-complexity linear-programming receiver for the case of binary coding and modulation 

Note that for the case of binary coding (SH = F2) and binary modulation, a lower-complexity LP may 
be developed. This LP is based on the use of the "parity polytope" of [24] which was applied to the 
case of linear-programming decoding of binary linear codes in ifTTTl . The new LP is defined as follows, 
where we omit the variables {wj t b} while introducing new variables /, for each i EU. 



LP5: Low-Complexity Joint Equalization and Decoding (Binary Coding and Modulation) 


Cost Function: 
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Constraints (Poly tope Q): These are given by (l25l) and (LTTl). together with 
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vi Eli, < jj < 1 , 
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Vjev^rcUj, \T\ odd, 5^/ f - 53 /i<l^l-i, 


(31) 






and 




Vi € W, ^ ?*,d = /i ■ 


(32) 


deX>: ip(d)=l 





The performance of LP5 is identical to that of LP4; this follows as a straightforward consequence of 
ITTTI Theorem 4]. 

VIII. Simulation Study: Binary Coded Transmission over Proakis B channel 

In this section we provide a simulation study of the linear-programming receiver of Section IVIII First 
we consider use of the binary [7, 4] Hamming code with BPSK modulation (constellation T = {— 1, +1}) 
over the Proakis B channel (H Chapter 10] with L = 2 and (hi, h 2 , h 3 ) = (1/V6, 2/y/6, l/y/6) (this 
channel is static and is normalized to unity power gain). For the binary [7, 4] Hamming code, we use the 
7x7 circulant parity-check matrix with first row (1110100). The minimum AWGN pseudoweight 
of the Hamming code with respect to this matrix is equal to 3, the code's minimum distance; this may 
be deduced by using the eigenvalue-based AWGN pseudoweight lower bound of [26, Theorem 1]. 

Due to the Proakis B channel however, the metric of importance in this context is not the minimum 
AWGN pseudoweight of the code, but the minimum pseudodistance of the system as defined by ( fT9l . 
The minimum pseudodistance of the system is d^ n = = 4/3, and the second smallest system 
pseudodistance is d^ = \f 7 2. A complete characterization of the corresponding error events is as follows. 
The pseudoconfiguration corresponding to the codeword c\ = (1 1 1 1 0) is at pseudodistance 
d^g = 4/3 from the pseudoconfiguration Ki corresponding tcQ 91,100 = 92,110 = 1 an( l 93,011 = 93, 111 = 



7 Note that we must have q l d = for d £ {000, 100} and q 2 d = for d {000, 010, 100, 110}, since the LP is constrained 
to recognize that the initial state of the channel is so = 00. 



94,011 = 94,101 = 95,010 = 95,101 = 96,010 = 96,ioi = 97,oio = 97,ioi = 1/2, for which 

2 2 4 2 2 

*" ( 76' "To' "T!' "71' ' ' Tr 
m = ( 7i' "71' ~7i' ~7I' °' °' 0) 

and 

v = (2/3, 2/3, 5/3, 1/3, 0, 0, 0) 

(the reader may verify using ( fl~9l ) that the pseudodistance is 4/3). The pseudoconfiguration corresponding 
to the codeword c\ is also at pseudodistance cr2 = \/2 from the pseudoconfiguration corresponding 
to the codeword C2 = (1101001) (in this case the pseudodistance is equal to the Euclidean 
distance between the corresponding modulated signals). Also, the pseudoconfiguration corresponding to 
the codeword C2 is at pseudodistance = \[2 from the pseudoconfiguration K2 corresponding to 
9i,ioo = 1. 92,oio = 93,ioi = 94,oio = 95,101 = 96,010 = 97,001 = 2/3 and 92,110 = 93,on = 94,ioi = 
95,oio = 96,ooi = 97,ioo = 1/3, for which 

2 2 2 2 2 

*" ( 76' "Te' "71' ' ' 7!' 7E } 
_ 2 2 2 2 2 
m " ( 7!' " sTe' "sTe' °' °' 371' 76 } 

and 

v = (2/3, 2/9, 2/9, 0, 0, 2/9, 2/3) . 

Similarly, the pseudoconfiguration corresponding to the codeword C3 = (0 1 1 1) is at 
pseudodistance =4/3 from the pseudoconfiguration ^3 corresponding to 91,000 = 92,000 = 1 and 

93,ooo = 93,100 = 94,010 = 94,100 = 95,010 = 95,101 = 96,010 = 96,101 = 97,010 = 97,101 = 1/2, for which 

4 4 4 2 2 

*" ( 7i' 7i' 7i' 7i' ~7i ) 
m = ( ^' ^' ^' 7i' ' °' 0) 

and 

v = (8/3, 8/3, 5/3, 1/3, 0, 0, 0) . 

The pseudoconfiguration corresponding to the codeword C3 is also at pseudodistance S3 = V2 from 
the pseudoconfiguration corresponding to the codeword C4 = (0 1 1 1 0), which in turn is at 
pseudodistance = y/2 from the pseudoconfiguration K4 corresponding to 91,000 = 1, 92,100 = 93,010 = 



94,101 — 95,010 — 96,101 — 97,110 — 2/3 and (72,000 — 93, 100 — 94,oio — 95,101 — 96, 110 — 97,on — 1/3, for 
which 

4 4 2 2 2 

4 8 2 9 9 

m = 7T7S> 7T7S> °> °> 



and 



V6' 3x/6' 3^6' ' ' 3^6' >/6 



« = (8/3, 4/3, 2/9, 0, 0, 2/9, 2/3) . 



Using this analysis, we may lower bound the frame error rate (FER) by 

Here the first term is due to the codewords C\ and C3, and the second term is due to the codewords 
C2 and C4. The bit error rate (BER) and frame error rate (FER) performance of the linear-programming 
receiver is shown in Figure [3j along with the lower bound on the FER given by d33l ). Here 500 reception 
errors were simulated for each value of E^/Nq. 

The measured pseudodistance "spectra" are shown in Figure [4] for four values of signal to noise ratio 
(SNR) spanning the simulation range; this provides a statistical record of the error events experienced by 
the receiver at simulated values of E^/Nq equal to 3 dB, 7 dB, 11 dB and 14 dB. Each spectrum also 
indicates, for each value of pseudodistance d, whether pseudoconfigurations at distance d from transmitter 
configurations consist of configurations only, non-configurations only, or both. Note that even at high 
SNR, no single error event dominates and that pseudoconfigurations as well as non-pseudoconfigurations 
contribute to receiver errors. Note also that for most transmitter configuration pairs whose pseudodistance 
is d, there also exist (configuration, non-configuration) pairs with pseudodistance d. It may be seen that 
as the SNR increases, error events at the smallest and second smallest pseudodistances, d^2 =4/3 (due 
to two error events involving non-configurations) and cr2 = \pi (due to two error events involving 
configurations, and two involving non-configurations), begin to somewhat dominate the pseudodistance 
spectrum; these are the error events discussed previously in this section involving codewords c\, C2, C3 
and C4. This predicts that for high SNR, the bound of (l33l) becomes reasonably tight; tighter bounds may 
be developed by taking into account the nearest-neighbour pseudoconfigurations (in the pseudodistance 
sense) of codewords other than c\, C2, C3 and C4. 

The complexity of LP decoding precludes the testing of long LDPC codes, however as a more practical 
example we test a low-density code of length 105 and rate 4/7, again over the Proakis B channel. The 



parity-check matrix consists of m = 45 rows and is equal to the right-circulant matrix 

( 1 if i-j€ {0,13,48,60} 
otherwise. 

It may be seen from Figure [3] that under LP reception, the low-density coded system provides a gain of 
approximately 4 dB over the Hamming coded system at a BER of 10~ 4 . For the case of the low-density 
coded system, the simulation estimated the system pseudodistance at d^ n = 36.80. 

Conclusions and Future Work 

A general linear-programming based communication receiver design technique has been presented. It 
was shown that the performance of such a receiver can be characterized sharply via the concepts of 
maximum likelihood certificate property, pseudoconfigurations and system pseudodistance. It is hoped 
that the results of this paper motivate further investigation into the use of system pseudodistance as a 
design tool for SPA as well as LP receivers. Certainly, system design could benefit from the development 
of efficient system pseudodistance spectrum evaluation at low SNR in order to provide performance 
prediction at high SNR, as discussed in ETTl . Another important investigation which is yet to be made 
is the application of efficient LP solvers such as the interior point methods of |[28l . ||29l , 11301 as well as 
the soft dual coordinate-ascent and sub-gradient based methods of IT3T1 which may be brought to bear 
on this problem in order to reduce the complexity of LP receivers. 
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Fig. 1. Illustration of the relationships involved in the proof of Theorem 15. II 




Fig. 2. Factor graph for coded transmissions with memoryless modulation over a frequency selective channel. The 
factor graph is illustrated for n = 7, m = 3, and the binary [7,4] Hamming code. Here y = {dijtfe, ••• ,dj} and 
<£ = {Pi Xi> X2> X3> > ^2, • • • , T7}. Also indicated on the graph are the relevant LP variables. The constraints of the LP 
(acting on these variables) may be read directly from the edges of the factor graph. 




Fig. 3. Bit error rate (BER) and frame error rate (FER) results for the linear-programming receiver which performs joint 
equalization and decoding. The plots are shown for the [7, 4] Hamming code and the [60, 105] low-density code with BPSK 
modulation over the Proakis B channel. For the case of the [7, 4] Hamming code, also plotted is the lower bound on the FER 
given by {33}. 
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Fig. 4. Measured spectra of error event pseudodistances for (top to bottom) Eb/No = 3 dB, 7 dB, 11 dB and 14 dB. The 
plots are for the case of [7, 4] Hamming coded transmission over the Proakis B channel. Each spectrum also indicates, for each 
value of pseudodistance d, whether pseudoconfigurations at distance d from transmitter configurations consist of configurations 
only, non-configurations only, or both. 



